变化自动编码器（VAE）最近已用于对复杂密度分布的无监督分离学习。存在许多变体，以鼓励潜在空间中的分解，同时改善重建。但是，在达到极低的重建误差和高度分离得分之间，没有人同时管理权衡。我们提出了一个普遍的框架，可以在有限的优化下应对这一挑战，并证明它在平衡重建时，它优于现有模型的最先进模型。我们介绍了三个可控的拉格朗日超级参数，以控制重建损失，KL差异损失和相关度量。我们证明，重建网络中的信息最大化等于在合理假设和约束放松下摊销过程中的信息最大化。

Graph Neural Networks (GNNs) have shown great potential in the field of graph representation learning. Standard GNNs define a local message-passing mechanism which propagates information over the whole graph domain by stacking multiple layers. This paradigm suffers from two major limitations, over-squashing and poor long-range dependencies, that can be solved using global attention but significantly increases the computational cost to quadratic complexity. In this work, we propose an alternative approach to overcome these structural limitations by leveraging the ViT/MLP-Mixer architectures introduced in computer vision. We introduce a new class of GNNs, called Graph MLP-Mixer, that holds three key properties. First, they capture long-range dependency and mitigate the issue of over-squashing as demonstrated on the Long Range Graph Benchmark (LRGB) and the TreeNeighbourMatch datasets. Second, they offer better speed and memory efficiency with a complexity linear to the number of nodes and edges, surpassing the related Graph Transformer and expressive GNN models. Third, they show high expressivity in terms of graph isomorphism as they can distinguish at least 3-WL non-isomorphic graphs. We test our architecture on 4 simulated datasets and 7 real-world benchmarks, and show highly competitive results on all of them.

This paper is devoted to the numerical resolution of McKean-Vlasov control problems via the class of mean-field neural networks introduced in our companion paper [25] in order to learn the solution on the Wasserstein space. We propose several algorithms either based on dynamic programming with control learning by policy or value iteration, or backward SDE from stochastic maximum principle with global or local loss functions. Extensive numerical results on different examples are presented to illustrate the accuracy of each of our eight algorithms. We discuss and compare the pros and cons of all the tested methods.

This paper describes the system developed at the Universitat Polit\`ecnica de Catalunya for the Workshop on Machine Translation 2022 Sign Language Translation Task, in particular, for the sign-to-text direction. We use a Transformer model implemented with the Fairseq modeling toolkit. We have experimented with the vocabulary size, data augmentation techniques and pretraining the model with the PHOENIX-14T dataset. Our system obtains 0.50 BLEU score for the test set, improving the organizers' baseline by 0.38 BLEU. We remark the poor results for both the baseline and our system, and thus, the unreliability of our findings.

This paper studies the infinite-width limit of deep linear neural networks initialized with random parameters. We obtain that, when the number of neurons diverges, the training dynamics converge (in a precise sense) to the dynamics obtained from a gradient descent on an infinitely wide deterministic linear neural network. Moreover, even if the weights remain random, we get their precise law along the training dynamics, and prove a quantitative convergence result of the linear predictor in terms of the number of neurons. We finally study the continuous-time limit obtained for infinitely wide linear neural networks and show that the linear predictors of the neural network converge at an exponential rate to the minimal $\ell_2$-norm minimizer of the risk.

Recent advances in deep learning models for sequence classification have greatly improved their classification accuracy, specially when large training sets are available. However, several works have suggested that under some settings the predictions made by these models are poorly calibrated. In this work we study binary sequence classification problems and we look at model calibration from a different perspective by asking the question: Are deep learning models capable of learning the underlying target class distribution? We focus on sparse sequence classification, that is problems in which the target class is rare and compare three deep learning sequence classification models. We develop an evaluation that measures how well a classifier is learning the target class distribution. In addition, our evaluation disentangles good performance achieved by mere compression of the training sequences versus performance achieved by proper model generalization. Our results suggest that in this binary setting the deep-learning models are indeed able to learn the underlying class distribution in a non-trivial manner, i.e. by proper generalization beyond data compression.

The combination of machine learning models with physical models is a recent research path to learn robust data representations. In this paper, we introduce p$^3$VAE, a generative model that integrates a perfect physical model which partially explains the true underlying factors of variation in the data. To fully leverage our hybrid design, we propose a semi-supervised optimization procedure and an inference scheme that comes along meaningful uncertainty estimates. We apply p$^3$VAE to the semantic segmentation of high-resolution hyperspectral remote sensing images. Our experiments on a simulated data set demonstrated the benefits of our hybrid model against conventional machine learning models in terms of extrapolation capabilities and interpretability. In particular, we show that p$^3$VAE naturally has high disentanglement capabilities. Our code and data have been made publicly available at https://github.com/Romain3Ch216/p3VAE.

在过去的几年中，神经网络（NN）从实验室环境中发展为许多现实世界中的最新问题。结果表明，NN模型（即它们的重量和偏见）在训练过程中的重量空间中的独特轨迹上演变。随后，这种神经网络模型（称为模型动物园）的人群将在体重空间中形成结构。我们认为，这些结构的几何形状，曲率和平滑度包含有关训练状态的信息，并且可以揭示单个模型的潜在特性。使用这种模型动物园，可以研究（i）模型分析的新方法，（ii）发现未知的学习动力学，（iii）学习此类人群的丰富表示形式，或（iv）利用模型动物园来用于NN权重和NN权重的生成模型偏见。不幸的是，缺乏标准化模型动物园和可用的基准可以显着增加摩擦，以进一步研究NNS人群。通过这项工作，我们发布了一个新颖的模型动物园数据集，其中包含系统生成和多样化的NN模型种群，以进行进一步研究。总共提出的模型动物园数据集基于八个图像数据集，由27个模型动物园组成，该模型动物园训练有不同的超参数组合，包括50'360唯一的NN型号以及其稀疏双胞胎，导致超过3'844'360收集的型号。 。此外，对于模型动物园数据，我们提供了对动物园的深入分析，并为多个下游任务提供了基准。该数据集可在www.modelzoos.cc上找到。

给定模型动物园的神经网络权重的学习表示是一个新兴而具有挑战性的领域，从模型检查到神经体系结构搜索或知识蒸馏，具有许多潜在的应用。最近，在模型动物园进行训练的自动编码器能够学习一个超代理，该代表体捕获了动物园中模型的内在和外在特性。在这项工作中，我们扩展了超代表，以供生成使用以采样新的模型权重。我们提出的是层损失归一化，我们证明，这是基于超代表拓扑生成高性能模型和几种采样方法的关键。使用我们的方法生成的模型是多种多样的，性能的，并且能够超过强大的基准，从而在下游任务上进行了评估：初始化，合奏采样和传递学习。我们的结果表明，通过超代理通过过度代理，知识聚集从模型动物园到新模型的潜力，从而为新的研究方向铺平了途径。

基于用法的保险已成为车辆保险的新标准；因此，找到使用保险人的驾驶数据的有效方法很重要。在车辆的行程摘要中应用异常检测，我们开发了一种方法，允许为每辆车辆得出“常规”和“特殊性”异常轮廓。为此，使用每辆车辆进行的每次旅行的异常检测算法来计算常规和特殊性异常得分。与相关车辆进行的其他旅行相比，前者测量了旅行的异常程度，而后者则与任何车辆进行的旅行相比，衡量了其异常程度。所得的异常得分向量用作常规和特殊性曲线。然后从这些配置文件中提取功能，我们为其研究索赔分类框架中的预测能力。使用真实数据，我们发现从车辆的特殊性概况提取的功能改善了分类。